Modeling downtime severity of telecommunication networks using discrete time Markov Chains

Analyzing Networked Automation Systems using Discrete Time Markov Chains

This paper investigates the application of probabilistic model checking to the analysis of Networked Automation Systems (NAS). Besides delay generating system structures due to shared component, asynchronously executed cyclic processes, and network transfers, hard bounds on response time have to be considered. To determine the response time, a modeling method for NAS is introduced and compared ...

Lecture 11: Discrete Time Markov Chains

where {Zn : n ∈ N} is an iid sequence, independent of initial state X0. If Xn ∈ E for all n ∈ N0, then E is called the state space of process X . We consider a countable state space, and if Xn = i ∈ E, then we say that the process X is in state i at time n. For a countable set E, a stochastic process {Xn ∈ E,n ∈ N0} is called a discrete time Markov chain (DTMC) if for all positive integers n ∈ ...

Discrete Time Markov Chains : Ergodicity Theory

Lecture 8: Discrete Time Markov Chains: Ergodicity Theory Announcements: 1. We handed out HW2 solutions and your homeworks in Friday’s recitation. I am handing out a few extras today. Please make sure you get these! 2. Remember that I now have office hours both: Wednesday at 3 p.m. and Thursday at 4 p.m. Please show up and ask questions about the lecture notes, not just the homework! No one cam...

Discrete Time Markov Chains 1 Examples

Example 1.1 (Gambler Ruin Problem). A gambler has $100. He bets $1 each game, and wins with probability 1/2. He stops playing he gets broke or wins $1000. Natural questions include: what’s the probability that he gets broke? On average how many games are played? This problem is a special case of the so-called Gambler Ruin problem, which can be modelled using a Markov chain as follows. We will b...

Lecture 10: Discrete Time Markov Chains